Presentation Handouts

• PDF versions are available online
  
  

• QR codes and website information
• Training handout and standard measures

About Ferris State

General Education Assessment

• Goal: Track and improve program effectiveness
  

  • Our program primarily consists of courses
  • Embedded course assignments are the most meaningful and valid measures of student learning.
  • General Education applies to all students
    

• Therefore, we are measuring student performance in General Education using embedded assessments for all students in all courses.

Project Scope

• In a typical semester, General Education at Ferris:
  

  • Offers over 400 different courses
  • Instructs over 5,000 individual students
  • Involves over 500 different faculty
  • Generates over 50,000 student credit hours
  • Is responsible for $20,000,000 in revenue

Project Resources

• Program coordinator (me, 50% release)
• General Education Committee (advisory, no release)
• Line item budget ($0)
• Dedicated staff (none)
• Other staff (Registrar and IR)

The Assessment Cycle

The Futile Cycle

A Better Model

“Assessment is not a spreadsheet – it is a conversation”
Irmeli Halinen

Reproducible assessment

Our goal is to create reproducible processes to increase transparency and encourage faculty engagement.

• ALL assessment resources are public-facing
  

  • Data reports (PDF and HTML)
  • Data analysis code (R)
  • Deidentified data files (csv)
    

• All released with MIT or CC-BY licenses
  
• All results are computationally reproducible and extensible

Striking a balance

Sustainable assessment

• The three keys to our program assessment strategy:
  
  

1. Simplification — reduce complexity
   
2. Standardization — reduce variance
3. Automation — reduce workload

Simplify: Core Competencies

• Eight competencies:
  

• Collaboration, Communication, Culture, Diversity, Natural Sciences, Problem Solving, Quantitative Literacy, Self and Society
  
• Each overseen by a faculty subcommittee (6-10 members)
  
  

1. Operational definitions
   
2. Goals (Hallmarks of a Bulldog)
   
3. Relevance statements

Simplify: Ferris Learning Outcomes

• Each competency has four learning outcomes
• Based upon what we want our graduating seniors to look like
• Derived from LEAP
• One outcome measured each semester
• Rolling two-year program cycle

Standardize: Rubrics

• Derived from VALUE rubrics
  
• General Education now has a uniform:
  
  

1. Scoring system (zero to four point scale)
   
2. Analysis interpretation (four threshold values)
   
3. Criterion leveling (expectations by course level)

Standardize: Measures

• Created in order to get more uniform data collection and summarization

• 14 different type of measures are defined
  
• Instructors can choose to use any one they wish
  
  

1. Exams
2. Products
3. Performances

Standardize: Registering Courses

• I need to know who is doing what each semester

• Who is collecting data?
• What competency is being measured?
• What outcome is being measured?
• What measure is being used?
• All tracked using a Google form

Standarize: Data Collection

• This is the biggest step forward for us

• Excel workbooks to collect and summarize student results
• SCORE, PREPOST, NORM, RUBRIC
• Common metadata supplied by instructors
• Enter student names, IDs, and appropriate evaluations
• A standaridized summary is automatically generated

Standardize: TracDat Assignments

• Seeking to minimize the need for faculty to log into TracDat

• Received as an email
• Copy and paste summary
• Select a few responses from drop-down menus
• Add a short reflection
• Attach the workbook and submit

Automate: Analysis of Student Data

• All workbooks are stored in the TracDat document repository

• Each semester’s data is in a separate folder
• Download all the workbooks
• An R script is used to process the workbooks
• All data is aggregated into one csv file
• Identifiable information is removed

Automate: Reporting

• Using R and bookdown (Rmarkdown)
  
• Reuseable, version-controlled code
  
• PDF reports for printing
  
• HTML reports for distribution and collaboration

Assessment in Action

• Five means of creating a dialog:
  
  

1. Print reports for dissemination
   
2. Online forums for extended discussions
   
3. OSF projects as the ultimate home base
   
4. GitHub repositories for reproducibility
   
5. Executive dashboard for overviews

Summary

• We are creating a collaborative assessment environment
  
• This is more about process than personality
  
• Focus is on making a resource rather than just reports

Questions?

• All of these resources are linked to from your handouts
  
• Licensed using either CC-BY 4.0 or MIT
  
• Feel free to contact me if you have questions or comments